1. Field of the Invention
The present invention relates in general to computers, and more particularly to a method, system, and computer program product for implementing a memory preserved cache in a storage subsystem to prevent data loss during a failover operation from a failed cluster to a surviving cluster of the storage subsystem, including a failsafe mechanism for the surviving cluster in the event of repetitive reboots.
2. Description of the Related Art
Storage devices such as disks are commonplace in today's society. Devices such as controllers control access to the storage devices in response to read and write requests. The storage controllers also mirror data to different storage devices and spread data amongst different storage devices for redundancy and backup purposes. Storage controllers may store data in accordance with one of several redundant array of independent disk (RAID) security levels. Generally, the higher the RAID level the greater the redundancy of the data storage. Pooled storage devices may be used to increase storage capacity and provide recovery and backup services.
Storage servers, such as an IBM Enterprise Storage Server (ESS), are also becoming commonplace. An IBM ESS storage server includes two clusters of processors and associated hardware. Typically, there are four storage controllers in each cluster. Each of the storage controllers controls multiple storage devices grouped in RAID arrays. In one environment, clients with Fiber Channel Host Bus Adapters (“HBAs”) are coupled via a Fiber Channel to a switch. The switch is also coupled to the Storage Server with Fiber Channel HBAs. There may be multiple storage servers per client. Each client is assigned or allocated storage “volumes” which are mapped to physical locations on storage devices that are grouped in RAID arrays. Consequently, clients make data access requests (reads and writes) to the storage server, for data within their allocated volumes, and the storage server accesses the mapped locations in cache storage to satisfy the requests or from disk if the data does not reside in cache storage.
A known IBM Enterprise Storage Server comprises two clusters of storage controllers with four storage controllers per cluster. Each cluster has its own cache (semiconductor) memory shared by all storage controllers in the cluster. Each cluster also has battery backed up nonvolatile storage (“NVS”) which is shared by all of the storage controllers in the cluster, although any one storage controller cannot use more than 25 percent of the total NVS allocated to the cluster. In addition, each storage controller has its own cache (disk) storage. The cache memory is used for rapid access to data inpaged from external storage to service read data access requests from memory and to provide buffering of modified data. All update requests are written to the associated cluster and backed up by the NVS on the mirrored cluster.
Occasionally, a cluster of the ESS may become partially or wholly inoperable, and may fail, in which case a failover process is implemented. The failover transfers operation of a failed component (in this case, the failed cluster) to a similar, redundant component (e.g., a surviving cluster) to ensure uninterrupted data flow. Data in NVS is preserved in the event of a power loss and reboot of an associated cluster. However, there are some time intervals where tracks of data reside solely in the cache (semiconductor) memory. One such time interval is immediately following a failover. Typically, some data to be stored on disk is in cache (with the remainder in NVS) on one cluster, with the reverse (data stored on cache in the first cluster stored in NVS of the second cluster, and data stored on NVS of the first cluster stored in cache of the second cluster) stored in the mirrored cluster. Following a failover of the first cluster, a reboot of the surviving cluster may cause data loss (as the copy stored in the first cluster's NVS is inaccessible and the mirrored copy is stored in the second cluster's cache, exposing it to the reboot operation). In cases of corrupt data, the surviving cluster may, however, continue to reboot, thus effecting the continued data flow in the storage subsystem.